Skip to content

feat: add eventually consistent metadata store for cross-replica coordination#229

Merged
alecthomas merged 1 commit intomainfrom
aat/metadatadb
Mar 27, 2026
Merged

feat: add eventually consistent metadata store for cross-replica coordination#229
alecthomas merged 1 commit intomainfrom
aat/metadatadb

Conversation

@alecthomas
Copy link
Copy Markdown
Collaborator

@alecthomas alecthomas commented Mar 27, 2026

This will initially be used for tracking a repo access histogram, but we
can also reuse this for a simpler version of the cache-invalidation (#38)
design.

Introduces the metadatadb package, providing generic redis-like data
structures (Scalar, Int, Set, Map, IntMap, List) backed by a pluggable storage
layer. Mutations apply locally immediately and sync periodically via a
lock-load-replay-store cycle. Includes memory and S3 backends, a
reusable test suite, and a soak test with monotonic invariant
verification.

Also extracts MinIO Docker test helpers into internal/minitest for
reuse across cache and metadatadb S3 tests.

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

@alecthomas alecthomas requested a review from a team as a code owner March 27, 2026 02:11
@alecthomas alecthomas requested review from stuartwdouglas and removed request for a team March 27, 2026 02:11
@alecthomas alecthomas enabled auto-merge (squash) March 27, 2026 02:11
@alecthomas alecthomas disabled auto-merge March 27, 2026 02:11
…dination

Introduces the metadatadb package, providing generic redis-like data
structures (Scalar, Int, Set, Map, List) backed by a pluggable storage
layer. Mutations apply locally immediately and sync periodically via a
lock-load-replay-store cycle. Includes memory and S3 backends, a
reusable test suite, and a soak test with monotonic invariant
verification.

Also extracts MinIO Docker test helpers into internal/minitest for
reuse across cache and metadatadb S3 tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alecthomas alecthomas merged commit d21a005 into main Mar 27, 2026
12 checks passed
@alecthomas alecthomas deleted the aat/metadatadb branch March 27, 2026 04:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant